Method for controlling data streams of a virtual session with multiple participants, collaboration server, computer program, computer program product, and digital storage medium

ABSTRACT

The invention relates to a method for controlling data streams of a virtual session with several participants who access at least access one application in the virtual session. A selected media processing and a signaling mode is used in a centralized process for each participant according to the individual requirements of the participant. The participants are represented by clients and the process is controlled by a server. The selection of the media processing and/or signaling mode is made on the basis of an evaluation of the terminal of the respective participant and the network bandwidth available to the respective participant. An evaluation scheme with a plurality of evaluation criteria is provided for the evaluation, and possible media processing and/or signaling modes are ascertained for the respective participant, said evaluation scheme being applied to each possible media processing and/or signaling mode. A media processing and/or signaling mode determined using a comparison of the evaluations for each possible media processing and/or signaling mode according to the applied evaluation scheme is selected for the respective participant. The invention further relates to a collaboration server, a computer program, a computer program product, and a digital storage medium with the corresponding functional features.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the United States national phase under 35 U.S.C.§371 of PCT international patent application no. PCT/EP2013/000524,filed on Feb. 22, 2013.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments relate to methods for controlling data streams of a virtualsession with multiple participants, a collaboration server, a computerprogram, a computer program product, and a digital storage medium.

2. Background of the Related Art

It is common practice to use a collaboration service or server for thecontrol of conferencing applications, and exchange of viewable contentand files between multiple participants. Today's collaboration serversare fine tuned to the nature of the participants, specific mediaformats, and/or signaling protocols. For example the collaborationservice sets up a connection for VoIP or ISDN technologies for audio, oraudio and video conferencing, and then only supports audio or audio andvideo. VoIP-solutions often only support SIP or H.323 as the signalingprotocol. On the other hand the so-called web collaboration servers,which offer for example shared desktop and whiteboard features, areknown, in their case the data exchange occurs frequently via the httpprotocol. As a further class web cast systems have establishedthemselves, in which exactly one video recording can be distributed to alarge number of passive participants via web/http protocols. Frequentlya content picture can be transmitted alongside, in which then, forexample a PowerPoint® presentation takes place in parallel.

In practice this means, for each type of media and client a separatecollaboration service is provided; they are installed, maintained andupdated in a server environment. If necessary, for a specific mediaand/or client type of a requesting participant where no collaborationservice is installed; this must be then first researched and installed,before a requesting participant can participate in a virtual meeting.

BRIEF SUMMARY OF THE INVENTION

It would be helpful to simplify the control of data streams of a virtualsession with multiple participants. It would also be helpful to reducethe administration costs for installation and maintenance of services ina server environment.

According to an aspect of the invention a method for controlling datastreams of a virtual session with several participants who at leastaccess one application in the virtual session is proposed whereby in acentralized process for each participant according to his/her individualrequirements uses a selected media processing and signaling mode, andwhereby the participants are represented by clients and the process iscontrolled by a server, whereby the choice of media processing and/orsignaling mode is chosen by an evaluation of a terminal of each of theparticipants and the respective network bandwidth available to theparticipant, whereby an evaluation scheme with a plurality of evaluationcriteria will be provided for the evaluation, whereby possible mediaprocessing and/or signaling modes are determined for a participant, saidevaluation scheme being applied to each possible media processing and/orsignaling mode, and whereby a media processing and/or signaling modedetermined using a comparison of the evaluations for each possible mediaprocessing and/or signaling mode according to the applied evaluationscheme is selected for each individual participant. A centralizedprocess within the meaning of the present invention is a process runningin an enclosed software and/or hardware instance. Through theapplication of the evaluation scheme it is also possible to select themedia processing and/or signaling mode with a best evaluation for therespective participant and thus select the optimal media processingand/or signaling mode for each participant.

BRIEF DESCRIPTION OF THE FIGURES

In the following the invention will be described based on preferredembodiments in detail and with the help of figures. Whereas

FIG. 1 is a schematic block diagram of a communication system with acollaboration service as a fundamental embodiment of the presentinvention;

FIG. 2 is a schematic block diagram of a media conversion instance inthe collaboration service of FIG. 1;

FIG. 3 is a schematic block diagram of a generic signal processing inthe collaboration service of FIG. 1;

FIG. 4 is a schematic flowchart of a process of a connection control inthe collaboration service of FIG. 1;

FIG. 5 is a schematic flowchart of a process for data exchange in theprocess of FIG. 4;

FIG. 6 is a schematic flow chart of a process for establishing protocolsand rules in the process of FIG. 4;

FIG. 7 is a schematic flow chart of a process for determining deviceparameters in the process of FIG. 6;

FIG. 8 is a schematic flow chart of a process for determining conversionrules in the process of FIG. 6.

The figures are purely schematic and not necessarily to scale. Thedrawing representations and descriptions thereof are meant as anexemplary illustration of the principle of the invention and should notlimit same in any way.

DETAILED DESCRIPTION OF THE INVENTION

According to an aspect of the invention a method for controlling datastreams of a virtual session with several participants who at leastaccess one application in the virtual session is proposed whereby in acentralized process for each participant according to his/her individualrequirements uses a selected media processing and signaling mode, andwhereby the participants are represented by clients and the process iscontrolled by a server, whereby the choice of media processing and/orsignaling mode is chosen by an evaluation of a terminal of each of theparticipants and the respective network bandwidth available to theparticipant, whereby an evaluation scheme with a plurality of evaluationcriteria will be provided for the evaluation, whereby possible mediaprocessing and/or signaling modes are determined for a participant, saidevaluation scheme being applied to each possible media processing and/orsignaling mode, and whereby a media processing and/or signaling modedetermined using a comparison of the evaluations for each possible mediaprocessing and/or signaling mode according to the applied evaluationscheme is selected for each individual participant. A centralizedprocess within the meaning of the present invention is a process runningin an enclosed software and/or hardware instance. Through theapplication of the evaluation scheme it is also possible to select themedia processing and/or signaling mode with a best evaluation for therespective participant and thus select the optimal media processingand/or signaling mode for each participant.

According to a preferred embodiment the applications include at leastsome of the following group, which contains:

-   -   audio and/or video conferences, particularly in a VoIP        environment;    -   data exchange between participants, in particular on the type of        shared desktop or whiteboard;    -   webcasting with or without transfer of content pictures.

According to a further preferred embodiment the media processing isselected from the group, which at least contains one adjustment of imageformats, one conversion of text into speech and one conversion oflanguage into text.

According to a further preferred embodiment the signaling mode isselected from the group, which contains SIP, H. 323, HTTP, Web/http andWebRTC.

According to a further preferred embodiment a generic signaling protocolis deployed, that is suitable for the abstraction of any signalingmodes, where a data stream will be abstracted from the virtual spaceinto the generic signaling protocol and then the generic signalingprotocol is transferred to the selected signaling mode for theparticipants. The generic signaling protocol is In particular asignaling protocol that belongs to the server. In addition, a datastream of the participant can be abstracted into the generic signalingprotocol and then the generic signaling protocol can be transmitted intoa signaling type required by the application.

According to a further preferred embodiment the evaluation systemprovides to award appropriate points for selected or all evaluationcriteria for the requirements, and add the score of all evaluationcriteria, in order to obtain an evaluation of a potential mediaupsampling, where in particular preference is given to a predeterminedweighting of the evaluation criteria.

According to a further preferred embodiment the evaluation criteria areselected from the group, which contains a browser resolution, an audioquality, a computing power on one side of the user and a conversioneffort on one side of a procedure, which is running the instance.

According to a further preferred embodiment, the centralized processmanages the virtual session.

According to a further preferred embodiment, the virtual meeting isshown to the participants in a virtual room.

According to a further aspect of the invention a collaboration server tocontrol data streams of a virtual session with multiple participants isproposed, which is designed and set up to execute the above describedprocedure.

With the process according to the invention participants can participatevia any kind of protocol, either VoIP/SIP, H323, web/http, WebRTC, orother. With this realized collaboration service or server any type ofparticipation and/or exchange of information can participate in a commonvirtual session or a virtual space, regardless of the type of theterminal equipment of the client, the technology used and the type ofmedia. The protocol technology is arbitrary, and the collaborationservice acts as a gateway, which offers always the best media and/orprotocol conversions. The collaboration service can replace a largenumber of specialist services and is therefore a universal solution, inwhich all possible media types can be used in an ongoing session andthus a more effective and more transparent virtual meeting may be held.It eliminates the need for many single solutions, and only a singlesystem is required, which can execute everything all at once.Furthermore the participants can also use their preferred client, andthe collaboration (server) adjusts the different technologiesaccordingly.

Further aspects of the present invention concern a computer program, acomputer program product, and a digital storage medium.

Further features, tasks, advantages and details of the present inventionwill be explained in more detail in the following description of exactimplementation examples and their drawings in the attached figures. Itgoes without saying that characteristics, tasks, advantages and detailsare transferable from individual design examples to others and areconsidered as disclosed in the context of the other embodiments, in sofar as this is not obviously absurd due to technical reasons or naturallaws. Design examples can be combined, and the combination can also beunderstood as an embodiment of the invention.

In the following the invention will be described based on preferredembodiments in detail and with the help of figures. Whereas

The figures are purely schematic and not necessarily to scale. Thedrawing representations and descriptions thereof are meant as anexemplary illustration of the principle of the invention and should notlimit same in any way.

A basic embodiment of the present invention is demonstrated forillustrative purposes below in the FIGS. 1 to 3. Accordingly FIG. 1 is aschematic block diagram of the communication system 100 with acollaboration service 130 as an embodiment of the present invention andFIGS. 2 and 3, are schematic block diagrams, which illustrates an innerstructure of a media conversion instance 135 and a generic signalprocessing 137 of the collaboration service 130.

According to the illustration in FIG. 1 in the communications system 100several client end devices 112-126 are connected with a collaborationservice 130.

The collaboration service 130 provides a virtual room 131 (“xyz”). Thevirtual room 131 can support a variety of applications, such as

-   -   audio conferencing    -   audio and video conferencing in real time (full duplex)    -   passive participation in live sessions (so-called “Webcast”        conferences)    -   exchange of viewable content (so-called “Shared Desktop”)    -   exchange of multi-media boards (so-called “Virtual Whiteboard”        or “Interactive Whiteboard”)    -   exchange of files (so-called “File Sharing”).

Other applications for data and/or media transmission in real-time andoffline are possible. The virtual room 131 can support a mix of theseapplications.

In addition, the collaboration service 130 includes a decision-makingunit 133, a generic media conversion instance 135, a generic signalprocessing 137, and network connections service or HTML/web service 139,which will be described later in more detail.

The client end devices 112-126 associated with the collaboration service130 include active and passive participants, which participate or intendto participate together in a conference or the like within the virtualroom 131. The active participants include an IP phone 112, an IPvideophone 114, an HTTP client 116 and an active network client 118.Passive participants include a passive viewer 122, a tablet PC 124 and asmartphone 126. While the active participants can send and receive data,the passive participants are only recipients of the data receptionmodus—the users of the passive participants are therefore observers.Each type of communication system, which enables a two-way transmissionof data to another communication system outside of the communicationfacility and allows further output devices for the transferred data to auser of the communication facility, can be used as an activeparticipant. Each type of communication system, which enables a one-waytransmission of data in the direction to another communication systemoutside of the communication facility and allows further output devicesfor the transferred data to a user of the communication facility, can beused as a passive participant. Therefore each active participant can beused as a passive participant.

The IP Phone 112 is a Voice over Internet Protocol (VoIP) enabledstandard telephone, which only supports voice and playback, andtransmits signals via SIP/RTP protocol.

The IP videophone terminal 114 is, for example, a video roaming system,which transmits via the SIP/RTP signals.

The http client 116 is an example of a remote computer, which supportsthe shared desktop, whiteboard, audio, and video conferencing, andsignals via http.

The active network client 118 is, for example, a remote computer, whichsupports the shared desktop, whiteboard, audio and video conferencing,and/or via http and WebRTC signals. In principle each type of computer,such as a workstation, a laptop/notebook, tablet/tablet PC, smartphoneetc. can be used as an active network client 118, which allows theidentification of at least one of the protocols VoIP, SIP, RTP, HTTP, orsupports another protocol for data transfer or any of the servicesWebRTC, HLS, shared desktop, whiteboard or some other service for datatransmission and display. In this respect the passive participants, whoencompass the passive viewer 122, the tablet PC 124 and the smartphone126, can act as active participants.

The passive viewer 122 is, for example, a remote computer, the shareddesktop, whiteboard, audio, and video conferencing, supported andsignaled via http, but currently only participates in observer mode(so-called “web cast”) in the conference in the virtual room 131 and isready to receive data via HLS (http live streaming).

The tablet computer 124 and the smartphone 126 are equipped with a webbrowser and capable for data transfer or reception via http/HLS; ifnecessary, WebRTC support is also intended.

The relatively well-known protocols and technical terms VoIP, SIP, RTP,HTTP, WebRTC, HLS, shared desktop, and whiteboard referred to the GermanWikipedia entries for Session Initiation Protocol, Real-Time TransportProtocol, Web RTC, HTTP Live Streaming Desktop Sharing, VOIP, andWhiteboarding, as called up on Nov. 18, 2012 whose full disclosure isincluded as a reference to this application.

It is understood that each of the participants 112-126 can represent awide variety of participants each identical or similar. It alsounderstood, that other type of participants may be present. It goeswithout saying that the participants support several services andsignaling protocols and can use and/or request these depending on theapplication. In terms of the supported services and the supportedsignaling protocols the invention is not specifically limited to thespecified selection.

As can be seen from the above description, participants 112-126 supportdifferent services and different signaling protocols.

The collaboration service 130 is designed such that it provides for eachof the participants 112-126 media processing and signaling of theindividual requirements for data exchanged via the virtual room 13. Tothis end the collaboration service 130 features a decision-making body133, which is also referred to as “Best Media Mapping Machine” (BMME) isa “Media Converter Instance” 135 (MCI), a Generic Signaling processing137 (GSH) and a HTML/web service 139. The decision-making authority 133makes a decision, how to process a respective media stream as much aspossible with the media converter instance 135, so that the participantscan participate as closely as possible in a meeting in the virtual room131. I.e., the decision-making authority 133 tries, depending onparticipant access or chosen participant terminal, to negotiate the bestpossible conversion of the various media types and provides availableconversions. The generic signaling processing 137 provides differentprotocol stacks to allow any clients, and HTML/web service 139, executeweb applications for the communication between the virtual room 131 andthe participants.

An inner structure of the media converter instance 135 is shown in FIG.2. Consequently, the media converter instance 135 demonstrates a widevariety of converting devices 201-299. For example, a converter device201 to transform a shared desktop JPG-image in a YUV-image stream, aconverter device 202 to convert a motion JPG-image sequence in aYUV-image stream, a converter device 211 for conversion of a YUV-videostream in VP8, a converter device 212 to transform a YUV-video stream inVP8 in H. 264/AVC, a TTS-converter device 298 to transform text streaminto an audio stream, an ASR-converter device 299 to transform an audiostream into a text stream. It is understood that the above described anddepicted instances in the figure are only an exemplary excerpt ofconverter devices from which media conversions are possible. The numberof possible conversion machines is also not limited to on a specificvalue such as 99. It is further to be noted that the media conversiondevice 135 allows not only a transcoding of a media type (e.g.pictures), but also a media type conversion (e.g. text to speech andvice versa).

For explanations of the relatively well known protocols and technicalterms JPG, Motion-JPG, YUV, VP8, H. 264/AVC, TTS, and ASR, the GermanWikipedia pages for JPEG, JPEG File Inter-change Format, Motion JPEG,YUV-Farbmodell, VP8, H.264, Speech Synthesis, and Speech Recognition, asopened on Nov. 8, 2012 and their full disclosure is included in thisapplication by reference.

Which conversion machine(s) is/are used, decides the decision-makingbody 133. The decision-making body 133 will assess the terminal and thenetwork bandwidth of the participant and decides which media streams andwill be converted how, and with what properties. The properties can alsochange: for example, the resolution of a video picture can decrease or ashared desktop screen size of 2540*1440 pixels can be reduced toVGA-H.264, etc.

The conversion rules are defined by configuration within thedecision-making body 133. Here, the rules each receive “points”. Indeed,multiple conversion rules can be possible and the system must decidewhich one to use. For example, a browser supports high or less highresolutions; the same applies to high or worse audio quality. Then onthe basis of the rules and the point allocation the best matching rule(“Matching Rule”) is used, i.e., the one rule which provides most of thepoints. This best rule also considers variables such as the of theperformance capabilities of the terminal device (for example, iOS hasfewer capabilities than a Window7 workstation) and the bandwidth, whichhas been monitored in passing (e.g., via RTCP), and also the conversionefforts on the server side (for example a conversion of H.264 in VP8 andvice versa would be “expensive” and would receive less points). Forexplanations of the relatively well-known protocol RTCP, reference ismade the German Wikipedia page for RTCP, as called up on Nov. 18, 2012,whose disclosure is included in this application by reference.

Construction of a generic signal processing 137 is illustrated in moredetail in FIG. 3. Consequently, the generic signal processing protocolstack 137 includes a streaming protocol stack 301 a signal protocolstack 302.

The streaming protocol stack 301 assigns the protocols WebRTC, RTP/SRTP,HTTP, and HLS.

The signal protocol stack 302 assigns the protocols SIP/SDP, http, andWebRTP/ROAP. ROAP (RTCWeb Offer/Answer Protocol) is a protocol for thenegotiation of media between browsers and other compatible devices.

The generic signal processing 137 also provides a generic signalingprotocol as an internal signaling protocol of the collaboration service130. For this the generic signal processing 137 converts a data streamfrom the virtual room or from a data stream originating from aparticipant into the generic signaling protocol. Then the generic signalprocessing 137 converts the generic signaling protocol into thesignaling mode selected for the participant to a signaling type requiredfor this application. Based on the streaming protocol stack and thesignaling protocol stack almost any adjustments to the signaling modeare possible in an efficient manner.

It is understood that the above described protocol stacks shown in thefigure are only an exemplary excerpt from the possible protocol stacks.The number is also not limited to the two shown. On the contrary, aprotocol stack should be made available for as many terminal devices aspossible at the starting point of the collaboration service 130.

The HTML/web service 139 now hosts web applications in the event that aparticipant with an active or passive (webcast) session will enter theroom through a web browser. It takes into consideration, whatresolutions the terminal supports, and the decision-making body 133 issupplied with the relevant information. The interpretation of theapplication as well as the media density adapts to the devicecapabilities. If the browser is actively involved in a meeting it sendsand receives data in real-time (for example, through WebRTC). However,if the browser is a passive participant in a meeting, it only receivesreal-time data, but does not send any. This can be achieved with theHLS-protocol for example. This ability also flows as a factor in thedecision of the decision-making body 133.

According to the above description participants can participate with thecollaboration service 130 via any kind of protocol, be it VoIP/SIP,H.323, web/http, WebRTC or other. With this the collaboration service130 allows any type of participation and/or exchange of information in acommon virtual session (virtual room 131), regardless of the type of theterminal equipment of the clients, the technology used and the type ofmedia. The protocol technology is arbitrary, and the collaborationservice 130 acts as a gateway, which offers always the best media and/orprotocol conversions. The collaboration service 130 can replace a largenumber of specialist services and is therefore a universal solution, inwhich all possible media types can be used in an ongoing session andthus a more effective and more transparent virtual meeting may be held.It eliminates the need for many single solutions, and only a singlesystem is required, which can execute everything all at once.Furthermore the participants can also use their preferred client, andthe collaboration service 130 (server) adjusts the differenttechnologies.

A practical approach to control data streams within the meaning of thepresent invention is now described by way of an exemplary embodiment,which is illustrated in FIGS. 4 to 8 in the form of schematic flowcharts. FIG. 4 is a schematic flow chart of a connection control process400 in the collaboration service of FIG. 1. FIG. 5 is a schematic flowchart of the process 500 for the data exchange in the virtual room,which is executed instead of a step 440 in the connection controlprocess 400. FIG. 6 is a schematic flow chart of a process 600 for thedetermination of protocols and rules, which is executed instead of astep 420 in the connection control process 400. FIG. 7 is a schematicflow chart of a subroutine 700 for the determination of the deviceparameters, which is executed instead of a step 610 in the determinationprocess 600; and FIG. 8 is a schematic flow chart of a subroutine 800,which is executed in the determination process 600 instead of a step 630or 670.

First, a connection control process 400 is described based on a flowchart shown in FIG. 4, which is performed by the collaboration service130 of FIG. 1. The connection control process 400 is a process orprocedure for control of data streams within the meaning of the presentinvention.

According to the illustration in FIG. 4 the connection control processis initiated after its start or call with a step 410 in which acollaboration request is received from an external client. Therequesting client can be, for example, but is not limited to, each ofthe participants 112-126 in FIG. 1. The step 410 is assigned to theHTML/web service 139 in FIG. 1.

Via a junction point 415, which is basically a function of a returnaddress in the process 400, the process 400 continues to a step 420, inwhich transmission protocols and conversion rules for data exchange arebeing used. Step 420 is illustrated in FIG. 5 as a process in greaterdetail, as commented in FIG. 4 by the Roman numeral “V”.

Then, in a step 430 based on the specified transmission protocolsestablished in step 420 a unidirectional or bidirectional data channelis established. The step 410 is again assigned to the HTML/web service139 in FIG. 1.

A junction point 435, which is basically a return address function inthe process 400, the process 400 continues to a step 440 in which thedata exchange in the virtual room 131 in FIG. 1 is controlled.

More specifically, data streams are controlled from the virtual room 131to the appropriate clients and (in the case of a bi-directionalconnection) from this to the virtual room 131. Step 440 is illustratedin FIG. 6 as a process in greater detail as commented in FIG. 4 by theRoman numeral “VI”.

The process 400 then continues to a step 450, in which will be assessedwhether the communication in step 430 via the established data channelis finished or not.

In the case of a positive evaluation in Step 450, i.e., if a terminationof the connection has been observed, in the next step 460 the datachannel is closed down orderly, and the process 400 ends or refers backto the calling process.

In the case of a negative evaluation in Step 450, i.e., no terminationof the connection has been observed, the next step 470 assesses whetheror not a change in parameters has occurred or not. A change inparameters is understood to be, for example, but not limited to amodification of the device parameters of the clients, a change in deviceparameters for other clients, which concerns a modification oftransmission protocols and/or conversion rules with respect to thoseclients, a modification in application parameters of an applicationrunning in a virtual room 131, or other parameters, which concern achange of transmission protocols and/or is using transformation rulesconcern, a change in a user application running in the virtual room 131,or other parameters, which could concern the communication with andprocessing at the client or the virtual room 131.

In the event of a positive assessment in Step 470, i.e., if a change inparameters has been observed, the process 400 jumps back to the branchpoint 415 in order to determine in Step 420 transmission protocols andconversion rules for this client and then continue further processingaccording to the above description.

In the event of a negative assessment in step 470, i.e., if no change inparameters has been observed, the process 400 jumps back to the branchpoint 435 in order to continue in Step 440 the control of the dataexchange and then continues the further processing according to theabove description.

The steps 450 to 470 are again assigned to the HTML/web service 139 inFIG. 1.

It is understood that the process 400 can run parallel or serial orsequentially for a wide variety of clients.

A data exchange control process 500 is now described with the help of aflowchart diagram shown in FIG. 5, which corresponds to the step 440 inthe connection control process 400 of FIG. 4 and is labeled there with“V”.

According to the illustration in FIG. 5, the process 500 leads after itsbeginning or its first call to a branch point 505. From here to the endof the left branch with steps 510 and 530, to receive data from aclient, or to a right branch with Steps 540 and 550, to transmit data tothe client. The right-hand branch (transmission branch) is alwaysprocessed; the left-hand branch (reception branch) is only processed inthe case of a bi-directional (i.e., full-duplex-) connection. Thedecision as to whether the left or the right branch is processed can bemade using call parameters or left and right branch are alternatelyprocessed at each call of the process 500.

In the left-hand branch of the process 500 in Step 510 the clientreceives a data stream. The step 510 is assigned to the HTML/web service139 in FIG. 1.

Then in Step 520 the data stream received in Step 510 is convertedaccording to a conversion rule as defined in step 420 (FIG. 4). The step520 is assigned to the media conversion instance 135 in FIG. 1.

Then, in Step 530 the data stream converted in Step 520 is shown in thevirtual room 131, so that it is visible to other participants. The step530 is assigned to the virtual room 131 in FIG. 1.

In the right-hand branch of the process 500 initially in Step 540 a datastream from the virtual room 131, which is to be sent to the client willbe converted with a conversion rule according to Step 420 (FIG. 4). Thestep 520 is again assigned to the media conversion instance 135 in FIG.1.

Secondly, in Step 550 the data stream converted in Step 540 is sent tothe client. The step 530 is again assigned to the HTML/web service 139in FIG. 1.

The left and right branch of the process 500 rejoins in a junction 555back together. Then the process 500 ends or refers back to the callingprocess.

Although branch point 505 is symbolized as an OR-branch, the left andthe right-hand branch also can be processed in parallel or in sequence.A query in order to assess if there is a bi-directional connection, canbe upstream from the left-hand branch, and will only be executed in theevent of a positive assessment, while in the case of a negativeassessment a direct jump to the junction point 555 occurs.

The steps 510 and 540 can also include a query, which, if no data streamis received from the client or transferred to the client directly jumpsto the junction point 555.

Based on a flow chart shown in FIG. 6 a protocol and rules determinationprocess 600 (short determination process 600) is described in step 420of FIG. 4, which corresponds to the connection control process 400 andis marked there as “VI”.

According to the illustration in FIG. 6 the process 600 is initiatedafter its start or the call of step 610, in which first the deviceparameters for clients are determined. The Roman numeral “VII”illustrates step 610 later on in FIG. 7 as a process in greater detailthan depicted in FIG. 6.

Then, in step 620 a source media type for transmissions to the client,i.e. a media type, which is requested by a queried application by theclient, is determined within the virtual room 131 in FIG. 1. The virtualroom 131 supplies this media type.

Then, in a step 630 a conversion rule for transmissions to client, i.e.,a rule for converting a data type to a data type usable or requested bythe client for use within the virtual room 131 in FIG. 1. The Romannumeral “VIII” illustrates step 630 later on in FIG. 8 as a process ingreater detail as depicted in FIG. 6.

Subsequently, in a step 640 a protocol stack for transmissions to theclient is selected.

In a step 650 it is then assessed whether or not the client is a passiveclient.

In the event of a positive assessment in step 650, i.e., if the clientis a passive client, process 600 jumps to a junction point 655, and thenprocess 600 ends or refers back to the calling process.

In the event of a negative assessment in step 650, i.e., if the clientis an active client, which requires a two-way data transmission, process600 continues to a step 660, in which a initially a source media typeprovided by one of the client data stream is determined. This media typeis delivered by the HTML/web service 139 in FIG. 1.

Then, in a step 670 a conversion rule for a reception of data streamsfrom the client is determined, i.e., a rule for converting a data typesupplied by the client into a data type required by the applicationwithin the virtual room 131 in FIG. 1. The step 670 corresponds to thestep 630 with reverse initial parameters and will be illustrated lateron in further detail as a process in FIG. 8.

Next, the process 600 leads to the junction point 655, in order to endthere.

A parameter determination process 700 is now described with the help ofa flowchart diagram shown in FIG. 7, which corresponds to the step 610in the determination process 600 of FIG. 6 and is labeled there with“VII”.

According to the illustration in FIG. 7 the process 700 is initiatedafter its start or call of step 710, in which first a receivedtransmission protocol for the client is determined. This transmissionprotocol is supplied by the HTML/web service 139 in FIG. 1.

Then in a step 720 an application type and/or a layout of a startingapplication on the client is determined. In a step 730 a screenresolution for the client is determined. In a step 740 a media density,which the client can process is determined. In a step 750, the audiosampling rate of the application is determined, and in Step 760 a deviceperformance of the clients is determined. Device performance canencompass, for example, but is not limited to, performance data of theprocessor, the memory, a graphic card, a sound card, attachedperipherals, etc.

Then the process 700 ends or refers back to the calling process.

It goes without saying that the above-described steps and parametersdetermined therein are an exemplary, non-exhaustive list. It is also notnecessary in every case to determine all of the parameters. In fact, thelist of parameters can be constrained to the practically most relevantcases depending on the type of application deployed and on thetechnology used.

A rules determination process 800 is now described with the help of aflowchart diagram shown in FIG. 8, which corresponds to the step 630 or670 in the determination process 600 of FIG. 6 and is labeled there with“VIII”. As already mentioned, the process 800 can be used for a rulesdetermination for a media conversion for transmission or reception, inwhich case only the call parameters are reversed.

According to the illustration in FIG. 8, the process 800 is initiatedafter its beginning or call by a step 810, which determines potentialconversion rules for the conversion of the source media type to thetarget media type.

Then, in a step 820 a designated number of possible evaluation criteriais used and evaluated for each possible conversion rule as determined bystep 810. For example, for each evaluation criterion an evaluation scoreof 0 up to a specified maximum points is assessed. Possible evaluationcriteria were already described above. It should be noted that thedevice parameters of the client, which were determined in step 610 inFIG. 6 (i.e., in the process 700 of FIG. 7) would be considered in theevaluation criteria. Equally the internal criteria of the collaborationservice 130 in FIG. 1 such as transformation cost can be considered inthe evaluation criteria.

Then in a step 830 the assessment for each possible conversion rule issummarized.

Finally, in a step 840 the conversion rule, which provides the bestassessment, is selected from the determined potential conversion rulesas determined by step 810. In other words, the conversion rule with thehighest score of all their evaluation criteria in total will beselected.

Then the process 800 ends or refers back to the calling process.

In summary, the invention provides for the control of data streams of avirtual session with multiple participants via a centralized processwhile using for each participant a selected media processing andsignaling mode according to his/her individual requirements. Acentralized process is a process, which runs on an enclosed softwareand/or hardware instance. The centralized process can be in particular acollaboration service or server. The invention can also be embodied by acomputer program, a computer program product or a digital storagemedium.

The present invention has been described above through the use ofpreferred embodiment and represented graphically. It should be noted,however, that the present invention is defined solely by the independentpatent claims and the above description of embodiments, modificationsand further developments serve only as an exemplary illustration. Notall elements described above are necessarily required for theapplication and implementation of the invention, as long as they are notintegrated in at least one independent claim as a mandatory feature.

We claim: 1-14. (canceled)
 15. A method for the control of virtualsession data streams with several participants, said participantsaccessing at least one application during a virtual session, comprising:for a plurality of participants in a centralized process, wherein eachparticipant is represented by a client, and wherein the process iscontrolled by a server; evaluating a plurality of evaluation criteriafor each participant for selecting an individual mode, said plurality ofevaluation criteria comprising evaluation of a terminal of eachparticipant and evaluation of a network bandwidth available to eachparticipant; comparing the evaluation criteria according to appliedevaluation matrices for possible media processing and signaling modes;and performing a media processing and signaling mode for eachparticipant according to the media processing and signaling evaluation.16. The method of claim 15, wherein the at least one application isselected from the group consisting of an audio conference, a videoconference, a shared desktop data exchange between participants, ashared whiteboard data exchange, and a webcast.
 17. The method of claim15, wherein the media processing is selected from the group consistingof adjustment of image formats, conversion of text into speech, andconversion of speech into text.
 18. The method of claim 15, wherein thesignaling mode is selected from the group consisting of sessioninitiation protocol (SIP), H.323, hypertext transfer protocol (HTTP),Web/http, and WebRTC.
 19. The method of claim 15, further comprising:deploying a generic signaling protocol for the abstraction of at leastone signaling mode; abstracting a data stream from a virtual room intothe generic signaling protocol; and transferring the generic signalingprotocol to the signaling mode for each participant.
 20. The method ofclaim 19, further comprising: abstracting the data stream into thegeneric signaling protocol; and transmitting the generic signalingprotocol into a signaling type required by the at least one application.21. The method of claim 15, further comprising: awarding points duringthe evaluation step for selected evaluation criteria; adding points ofall evaluation criteria; and obtaining an evaluation of a potentialmedia upsampling.
 22. The method of claim 21, wherein the evaluationcriteria are selected from the group consisting of a browser resolution,an audio quality, a computing power, and a conversion effort.
 23. Themethod of claim 15, wherein the centralized process manages the virtualsession.
 24. The method of claim 15, wherein the virtual session ispresented to the participants in a virtual room.
 25. A collaborationserver to control the data streams of a virtual session with multipleparticipants, said collaboration server executing steps comprising: fora plurality of participants in a centralized process, wherein eachparticipant is represented by a client, and wherein the process iscontrolled by a server; evaluating a plurality of evaluation criteriafor each participant for selecting an individual mode, said plurality ofevaluation criteria comprising evaluation of a terminal of eachparticipant and evaluation of a network bandwidth available to eachparticipant; comparing the evaluation criteria according to appliedevaluation matrices for possible media processing and signaling modes;and performing a media processing and signaling mode for eachparticipant according to the media processing and signaling evaluation.26. A computer program containing computer program instructions that,when executed, performs the method of claim
 15. 27. A computer-readablestorage medium comprising the computer program of claim 26.